{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "cbb830a8",
   "metadata": {},
   "source": [
    "# 支持向量机 SVM(Support Vector Machine)\n",
    "        支持向量机是一种监督学习算法\n",
    "        *“支持向量”就是离分隔超平面最近的那些点\n",
    "        * “机”就是表示一种算法，而不是表示机器\n",
    "        \n",
    "**将二维数据变成三维数据的过程，称为将数据投射到高维空间，这正是 SVM 算法的核函数功能**切分数据的平面称为超平面。\n",
    "* 在 SVM 中，最普遍的把两种数据投射到高维空间的方法：\n",
    "    1. **多项式内核**（Polynomial Kernel）\n",
    "    2. **径向基内核**（Radial Basis Fuction kernel,RBF）\n",
    "\n",
    "SVM 是一个二类分类器，它的目标是找到一个超平面，使两类数据里超平面越远越好，而对新的数据的分类更准确，也就是使分类器更加健壮。\n",
    "\n",
    "支持向量（Support Vector）就是离分隔超平面最近的那些点。\n",
    "\n",
    "寻找最大间隔就是寻找最大化支持向量到分隔超平面的距离，在此条件下求出分隔超平面。\n",
    "\n",
    "**求解“决策面”的过程，就是最优化。**\n",
    "        \n",
    "        分隔间隙越大也好，这样分出来的特征精确性更高，容错空间也更大。\n",
    "        \n",
    "        保证间隔尽可能的大就是保证我们的分类器误差尽可能地小，尽可能的健壮。\n",
    "        \n",
    "***\n",
    "<br>\n",
    "\n",
    "## 核函数\n",
    "\n",
    "        对于非线性的情况，SVM 的处理方法就是选择一个核函数，将数据映射到高维空降，来解决在原始空间中线性不可分的问题。\n",
    "        \n",
    "        核函数的价值在于它虽然是将特征从低维转换到高维，但是核函数事先在低位上进行计算，而将实质上的分类效果表现在高维上。\n",
    "        \n",
    "        核函数的作用就是保证在不增加算法复杂度的情况下，将完全不可分的问题转化为 可分 或 达到近似可分 的状态。\n",
    "        \n",
    "在 SVM 中，核函数的存在，使得运算仍然在低维空间进行，避免了在高位空间中复杂运算的时间消耗。\n",
    "\n",
    "\n",
    "\n",
    "### 松弛变量\n",
    "SVM 另一个巧妙之处是加入了一个松弛变量来处理样本数据可能存在的噪声问题。SVM 允许数据点在一定程度上，对超平面有所偏离，这个偏移量就是 SVM 算法中可以设置的 outlier 值（离群值）。\n",
    "\n",
    "松弛变量的加入使得 SVM 并非仅仅是追求局部最优，而是从样本数据分布的全局出发，统筹考虑。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1362428e",
   "metadata": {},
   "source": [
    "# ❉SVM 的核函数选择和参数调整\n",
    "**想要SVM 用的好，会调参真的很重要。**\n",
    "\n",
    "https://blog.csdn.net/ztf312/article/details/98594359\n",
    "https://zhuanlan.zhihu.com/p/37189815"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "3180863e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 因为数据集涉及种族问题，所以在sklearn 1.2版本中被移除。\n",
    "# 报错的解决办法也在报错中给出了。\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "data_url = \"http://lib.stat.cmu.edu/datasets/boston\"\n",
    "raw_df = pd.read_csv(data_url, sep=\"\\s+\", skiprows=22, header=None)\n",
    "data = np.hstack([raw_df.values[::2, :], raw_df.values[1::2, :2]])\n",
    "target = raw_df.values[1::2, 2]\n",
    "\n",
    "# 这里我们的版本还是1.1.1,可以正常使用\n",
    "# from sklearn.datasets import load_boston\n",
    "# boston=load_boston()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "c3514fe8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[6.3200e-03, 1.8000e+01, 2.3100e+00, ..., 1.5300e+01, 3.9690e+02,\n",
       "        4.9800e+00],\n",
       "       [2.7310e-02, 0.0000e+00, 7.0700e+00, ..., 1.7800e+01, 3.9690e+02,\n",
       "        9.1400e+00],\n",
       "       [2.7290e-02, 0.0000e+00, 7.0700e+00, ..., 1.7800e+01, 3.9283e+02,\n",
       "        4.0300e+00],\n",
       "       ...,\n",
       "       [6.0760e-02, 0.0000e+00, 1.1930e+01, ..., 2.1000e+01, 3.9690e+02,\n",
       "        5.6400e+00],\n",
       "       [1.0959e-01, 0.0000e+00, 1.1930e+01, ..., 2.1000e+01, 3.9345e+02,\n",
       "        6.4800e+00],\n",
       "       [4.7410e-02, 0.0000e+00, 1.1930e+01, ..., 2.1000e+01, 3.9690e+02,\n",
       "        7.8800e+00]])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d7930a81",
   "metadata": {},
   "source": [
    "#### mgrid\n",
    "功能：返回多维结构，常见的如2D图形，3D图形\n",
    "\n",
    "> np.mgrid[ 第1维，第2维 ，第3维 ， …] \n",
    "\n",
    "第n维的书写形式为：\n",
    "\n",
    "> a:b:c\n",
    "\n",
    "c表示步长，为实数表示间隔；该为长度为[a,b),左闭右开\n",
    "\n",
    "或：\n",
    "\n",
    "> a:b:cj\n",
    "\n",
    "cj表示步长，为复数表示点数；该长度为[a,b]，左闭右闭"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "cef73965",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "a,b,c = np.mgrid[-1:1:2j,-2:2:2j,-3:3:5j]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "b699a161",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[-1.0,\n",
       " -1.0,\n",
       " -1.0,\n",
       " -1.0,\n",
       " -1.0,\n",
       " -1.0,\n",
       " -1.0,\n",
       " -1.0,\n",
       " -1.0,\n",
       " -1.0,\n",
       " 1.0,\n",
       " 1.0,\n",
       " 1.0,\n",
       " 1.0,\n",
       " 1.0,\n",
       " 1.0,\n",
       " 1.0,\n",
       " 1.0,\n",
       " 1.0,\n",
       " 1.0]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(a.flat)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "025a7d3e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[[-2., -2., -2., -2., -2.],\n",
       "        [ 2.,  2.,  2.,  2.,  2.]],\n",
       "\n",
       "       [[-2., -2., -2., -2., -2.],\n",
       "        [ 2.,  2.,  2.,  2.,  2.]]])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "84f985a7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[[-3. , -1.5,  0. ,  1.5,  3. ],\n",
       "        [-3. , -1.5,  0. ,  1.5,  3. ]],\n",
       "\n",
       "       [[-3. , -1.5,  0. ,  1.5,  3. ],\n",
       "        [-3. , -1.5,  0. ,  1.5,  3. ]]])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "c"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "139d1b20",
   "metadata": {},
   "outputs": [],
   "source": [
    "np.mgrid[-1:1:2j,-2:2:2j,-3:3:5j]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "bf3149d0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-2. , -3. ],\n",
       "       [-2. , -1.5],\n",
       "       [-2. ,  0. ],\n",
       "       [-2. ,  1.5],\n",
       "       [-2. ,  3. ],\n",
       "       [ 2. , -3. ],\n",
       "       [ 2. , -1.5],\n",
       "       [ 2. ,  0. ],\n",
       "       [ 2. ,  1.5],\n",
       "       [ 2. ,  3. ],\n",
       "       [-2. , -3. ],\n",
       "       [-2. , -1.5],\n",
       "       [-2. ,  0. ],\n",
       "       [-2. ,  1.5],\n",
       "       [-2. ,  3. ],\n",
       "       [ 2. , -3. ],\n",
       "       [ 2. , -1.5],\n",
       "       [ 2. ,  0. ],\n",
       "       [ 2. ,  1.5],\n",
       "       [ 2. ,  3. ]])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.stack((b.flat,c.flat),axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "9f004dc2",
   "metadata": {},
   "outputs": [],
   "source": [
    "x, y =np.mgrid[-1:1:2j,-2:2:2j]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "88b00079",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-1., -1.],\n",
       "       [ 1.,  1.]])"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "73f7210d",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "479f4886",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f20cb720",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e19029d9",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "767ce7e9",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "38827361",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bef81253",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
